Systems and methods for deducing user information from input device behavior

ABSTRACT

User selections entered in the media application or any user input device behavior with user devices may be recorded as clickstream data. The clickstream data may be used to deduce information about the user or a media item being consumed. A user may be profiled based on his or her input device behavior using a plurality of time-stamped indicators. A degree of user interest may be determined based on a time period between the time-stamped indicators, the number of time-stamped indicators within a period of time, and/or a type of user action.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 13/096,415, filed Apr. 28, 2011, now pending. Thecontents of which are hereby incorporated by reference herein in itsentirety.

BACKGROUND OF THE INVENTION

This invention relates generally to interactive media guidanceapplications, and more particularly, to systems and methods for deducinguser information from input device behavior.

Users can generally consume media via a variety of media devices. Duringthe consumption of media or interaction with media devices, userbehavior may be tracked by one or more applications or components thatcan capture and communicate clickstream information. A clickstream maybe a recording or a log of user selections or activity on a media devicesuch as during the consumption of a media item or during the interactionwith a media application. Analysis of clickstream data points may beperformed to assess media use, including: user media preferences, mediausage trends and patterns, audience measurements and/or characteristics,usage of certain features with a media device or applications running onthe media device. The analysis of clickstream data remains an area ofstudy.

Marketing professionals have clamored for clickstream data since the1990s. However, various problems continue to exist related toclickstream data. For example, there is a lack of quality specificationsthat define exactly which clicks to capture from a remote controller,mouse, or other input device. It's often not clear what contextinformation a user is viewing or clicking about when clickstream data isbeing gathered. There are concerns with user privacy related to gatheredinformation. There are also technical challenges caused by limitedbandwidth which has hobbled clickstream deployment and analysis.

Many systems focus on raw clickstream data points to predict informationabout a user. However, raw stream data is limited in the types ofinformation provided. Some systems use a single clickstream data pointto make an inference about a user. However, such anecdotal data can benoisy because the user may have pressed a button by mistake. Inaddition, single data points may not be as informative as data observedor aggregated over time. Accordingly, there is a need for a system thatdoes not rely on raw data points, but rather, examines other types orforms of clickstream data.

To make inferences, clickstream data generally relies on media contentthat has been labeled, i.e., content that has been identified asbelonging to a certain category or having a certain attribute. Thelabeling provides information related to the content of the media item,or portions thereof, such that an inference can be made about the userbased on the labeling of the media content. However, labeled content isexpensive and time consuming to create and maintain because humaneditors are typically required to physically view the content tocategorize (i.e., label) it. Accordingly, there is a need for a systemthat can label media content without requiring human intervention.

Additionally, media content is generally categorized into genres orother categories based on manual input from human editors. For example,a content provider may hire an editor to identify and categorize contentbased on, for example, genre. Alternatively, a marketing group study maybe conducted to identify the kinds of users for which a media program ismost suited. These methods are costly and time consuming. Accordingly,there is a need for a system that can create such categorical data formedia content without a formal marketing study or human editor to createthe data.

SUMMARY OF THE INVENTION

The present application discloses systems and methods that addressdeficiencies in the prior art by determining user and/or media contentcharacteristics based on clickstream data gathered from one or moremedia devices that are associated with one or more media users.

There are many types of user devices that may be used to consume and/orinteract with media items. Examples of such devices may includeaudiovisual devices, handheld portable devices, computers, televisions,personal communication devices, and other devices capable of presentingor supplying media. Media (e.g., a media item) may include music,television programming, movies, games, news, internet based mediacontent, videos, recordings, and other types of media. Media devices mayinclude a media application which may be used to identify, display,and/or access media items or content. For example, an interactive mediaapplication may be used with a set-top box, television, monitor or otherdisplay device for identifying and accessing television programs,interactive games, movies, music, or other types of media. In anotherexample, a media application may provide a user interface foridentifying and selecting media items or media content from mediaproviders and suppliers for consumption via a handheld device, computer,telephone, set-top box, television or other suitable device.

According to one aspect of the disclosure, a clickstream application maybe partially or entirely implemented on a user device or a remote serverfor gathering clickstream data. The clickstream application may besoftware and/or hardware, and configured to gather and/or analyzeclickstream data.

The clickstream application may record user selections entered via themedia application or any user input device behavior with user devices asclickstream data. Such clickstream data may be stored on the device, orat a remote location, and analyzed on a real time basis (e.g., withinabout 5 seconds or less), or at a later time, to determine media trendsand media device usage.

According to one aspect of the disclosure, the clickstream applicationmay profile a user based on his/her input device behavior. Theclickstream application may receive a plurality of time-stampedindicators based on user actions over a period of time with a mediadevice while interacting with a media item. Each of the time-stampedindicators may be associated with a type of user action with the mediadevice. The clickstream application may measure one or more time periodsbetween the time-stamped indicators and the number of time-stampedindicators within the period of time. Based on at least one time periodbetween the time-stamped indicators, the number of time-stampedindicators within the period of time, and at least one type of useraction, the clickstream application may determine a degree of userinterest in a portion of the media item.

According to another aspect of the disclosure, the clickstreamapplication may identify portions of a media item based at least on auser's interaction and/or behavior with an input device. For example,certain user activity may be identified as being related to an event,such as extraordinary or salient moment in a television program or othermedia item. The clickstream application may receive a plurality oftime-stamped indicators based on user actions over a period of time witha media device while interacting with a media item. Each of thetime-stamped indicators may be associated with a type of user actionwith the media device.

The clickstream application may define a plurality of behavior patternsbased on time-stamped indicators. Each behavior pattern may beassociated with a set of user actions. As the user interacts with themedia item, the clickstream application identifies an event associatedwith the media item by detecting a behavior pattern or change inbehavior pattern. In some features, the clickstream applicationidentifies an event associated with a media item by detecting one of thedefined behavior patterns as the user interacts with the media item.

According to yet another aspect of the invention, a clickstreamapplication may generate information about a media item. For example,users may be grouped based on their respective clickstream activityduring the consumption of one or more media items. Based on the usergrouping, the clickstream application may derive information about amedia item of content being consumed by a user of the group, such asgenre information, or parental ratings, and the like. The clickstreamapplication may receive time-stamped indicators during the consumptionof a media item from a plurality of users. Each of the time-stampedindicators may be associated with a type of user action with a usermedia device. Based at least in part on the received time-stampedindicators from each user media devices, the clickstream application mayclassify users into a plurality of groups. The clickstream applicationmay also derive information about the media item based in part on afirst characteristic shared among at least two of the users belonging toa first group of users.

The methods and systems described herein may be applied to any type ofmedia device in which an application, interface, or component isprovided for accessing media content, and which is capable of capturingclickstream information alone or in combination with other relateddevices.

A clickstream includes one or more of various types of data based onuser interactions or actions with a media device such as, withoutlimitation: a keyboard stroke, mouse click, joystick command, keypaddepression, voice command, touchscreen touch, haptic interaction,gesture input, remote control key initiation, device movement, usermovement, user expression, fast-forward command, rewind command, pause,stop, play, volume up, volume down, set-top box control command, and thelike.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and advantages of the invention will beapparent upon consideration of the following detailed description, takenin conjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 shows a display screen using a grid format that may be used toprovide guidance for various types of media;

FIG. 2 shows a display screen using a mosaic that may be used to provideguidance for various types of media;

FIG. 3 shows a user equipment device according to an illustrative aspectof the application;

FIG. 4 shows a simplified diagram of an illustrative interactive mediasystem;

FIG. 5 shows a flow diagram of a method for deducing user informationfrom user behavior;

FIG. 6 shows a flow diagram of a method for identifying a portion of amedia item;

FIG. 7 shows a flow diagram of a method for generating information abouta media item;

FIG. 8 shows an exemplary clickstream timeline;

FIG. 9 shows two example clickstream logs from two different users;

FIG. 10 shows another two example clickstream logs from two users;

FIG. 11 shows example clickstream logs from users in relation to a videoitem; and

FIG. 12 shows another example clickstream logs from users in relation toa media item.

DETAILED DESCRIPTION OF EMBODIMENTS

The amount of media available to users in any given media deliverysystem can be substantial. Consequently, many users desire a form ofmedia guidance through an interface that allows users to efficientlynavigate media selections and easily identify media that they maydesire. An application which provides such guidance is referred toherein as an interactive media guidance application or, sometimes, amedia guidance application or a guidance application.

Interactive media guidance applications may take various forms dependingon the media for which they provide guidance. One typical type of mediaguidance application is an interactive television program guide.Interactive television program guides (sometimes referred to aselectronic program guides) are well-known guidance applications that,among other things, allow users to navigate among and locate many typesof media content including conventional television programming (providedvia traditional broadcast, cable, satellite, Internet, or other means),as well as pay-per-view programs, on-demand programs (as invideo-on-demand (VOD) systems), Internet content (e.g., streaming media,downloadable media, Webcasts, etc.), and other types of media or videocontent. Guidance applications also allow users to navigate among andlocate content related to the video content including, for example,video clips, articles, advertisements, chat sessions, games, etc.Guidance applications also allow users to navigate among and locatemultimedia content. The term multimedia is defined herein as media andcontent that utilizes at least two different content forms, such astext, audio, still images, animation, video, and interactivity contentforms. Multimedia content may be recorded and played, displayed oraccessed by information content processing devices, such as computerizedand electronic devices, but can also be part of a live performance. Itshould be understood that the invention embodiments that are discussedin relation to media content are also applicable to other types ofcontent, such as video, audio and/or multimedia.

With the advent of the Internet, mobile computing, and high-speedwireless networks, users are accessing media on personal computers (PCs)and other devices on which they traditionally did not, such as hand-heldcomputers, personal digital assistants (PDAs), mobile telephones, orother mobile devices. On these devices users are able to navigate amongand locate the same media available through a television. Consequently,media guidance is necessary on these devices, as well. The guidanceprovided may be for media content available only through a television,for media content available only through one or more of these devices,or for media content available both through a television and one or moreof these devices. The media guidance applications may be provided ason-line applications (i.e., provided on a web-site), or as stand-aloneapplications or clients on hand-held computers, PDAs, mobile telephones,or other mobile devices. The various devices and platforms that mayimplement media guidance applications are described in more detailbelow.

One of the functions of the media guidance application is to providemedia listings and media information to users. FIGS. 1-2 showillustrative display screens that may be used to provide media guidance,and in particular media listings. The display screens shown in FIGS. 1-2may be implemented on any suitable device or platform. While thedisplays of FIGS. 1-2 are illustrated as full screen displays, they mayalso be fully or partially overlaid over media content being displayed.A user may indicate a desire to access media information by selecting aselectable option provided in a display screen (e.g., a menu option, alistings option, an icon, a hyperlink, etc.) or pressing a dedicatedbutton (e.g., a GUIDE button) on a remote control or other user inputinterface or device. In response to the user's indication, the mediaguidance application may provide a display screen with media informationorganized in one of several ways, such as by time and channel in a grid,by time, by channel, by media type, by category (e.g., movies, sports,news, children, or other categories of programming), or otherpredefined, user-defined, or other organization criteria.

FIG. 1 shows illustrative grid program listings display 100 arranged bytime and channel that also enables access to different types of mediacontent in a single display. Display 100 may include grid 102 with: (1)a column of channel/media type identifiers 104, where each channel/mediatype identifier (which is a cell in the column) identifies a differentchannel or media type available; and (2) a row of time identifiers 106,where each time identifier (which is a cell in the row) identifies atime block of programming. Grid 102 also includes cells of programlistings, such as program listing 108, where each listing provides thetitle of the program provided on the listing's associated channel andtime. With a user input device, a user can select program listings bymoving highlight region 110. Information relating to the program listingselected by highlight region 110 may be provided in program informationregion 112. Region 112 may include, for example, the program title, theprogram description, the time the program is provided (if applicable),the channel the program is on (if applicable), the program's rating, andother desired information.

In addition to providing access to linear programming provided accordingto a schedule, the media guidance application also provides access tonon-linear programming which is not provided according to a schedule.Non-linear programming may include content from different media sourcesincluding on-demand media content (e.g., VOD), Internet content (e.g.,streaming media, downloadable media, etc.), locally stored media content(e.g., video content stored on a digital video recorder (DVR), digitalvideo disc (DVD), video cassette, compact disc (CD), etc.), or othertime-insensitive media content. On-demand content may include bothmovies and original media content provided by a particular mediaprovider (e.g., HBO On Demand providing “The Sopranos” and “Curb YourEnthusiasm”). HBO ON DEMAND is a service mark owned by Time WarnerCompany L. P. et al. and The Sopranos and Curb Your Enthusiasm aretrademarks owned by the Home Box Office, Inc. Internet content mayinclude web events, such as a chat session or Webcast, or contentavailable on-demand as streaming media or downloadable media through anInternet web site or other Internet access (e.g. FTP).

Grid 102 may provide listings for non-linear programming includingon-demand listing 114, recorded media listing 116, and Internet contentlisting 118. A display combining listings for content from differenttypes of media sources is sometimes referred to as a “mixed-media”display. The various permutations of the types of listings that may bedisplayed that are different than display 100 may be based on userselection or guidance application definition (e.g., a display of onlyrecorded and broadcast listings, only on-demand and broadcast listings,etc.). As illustrated, listings 114, 116, and 118 are shown as spanningthe entire time block displayed in grid 102 to indicate that selectionof these listings may provide access to a display dedicated to on-demandlistings, recorded listings, or Internet listings, respectively. Inother embodiments, listings for these media types may be includeddirectly in grid 102. Additional listings may be displayed in responseto the user selecting one of the navigational icons 120. (Pressing anarrow key on a user input device may affect the display in a similarmanner as selecting navigational icons 120.)

Display 100 may also include video region 122, advertisement 124, andoptions region 126. Video region 122 may allow the user to view and/orpreview programs that are currently available, will be available, orwere available to the user. The content of video region 122 maycorrespond to, or be independent from, one of the listings displayed ingrid 102. Grid displays including a video region are sometimes referredto as picture-in-guide (PIG) displays. PIG displays and theirfunctionalities are described in greater detail in Satterfield et al.U.S. Pat. No. 6,564,378, issued May 13, 2003 and Yuen et al. U.S. Pat.No. 6,239,794, issued May 29, 2001, which are hereby incorporated byreference herein in their entireties. PIG displays may be included inother media guidance application display screens of the presentinvention.

Advertisement 124 may provide an advertisement for media content that,depending on a viewer's access rights (e.g., for subscriptionprogramming), is currently available for viewing, will be available forviewing in the future, or may never become available for viewing, andmay correspond to or be unrelated to one or more of the media listingsin grid 102. Advertisement 124 may also be for products or servicesrelated or unrelated to the media content displayed in grid 102.Advertisement 124 may be selectable and provide further informationabout media content, provide information about a product or a service,enable purchasing of media content, a product, or a service, providemedia content relating to the advertisement, etc. Advertisement 124 maybe targeted based on a user's profile/preferences, monitored useractivity, the type of display provided, or on other suitable targetedadvertisement bases.

While advertisement 124 is shown as rectangular or banner shaped,advertisements may be provided in any suitable size, shape, and locationin a guidance application display. For example, advertisement 124 may beprovided as a rectangular shape that is horizontally adjacent to grid102. This is sometimes referred to as a panel advertisement. Inaddition, advertisements may be overlaid over media content or aguidance application display or embedded within a display.Advertisements may also include text, images, rotating images, videoclips, or other types of media content. Advertisements may be stored inthe user equipment with the guidance application, in a databaseconnected to the user equipment, in a remote location (includingstreaming media servers), or on other storage means or a combination ofthese locations. Providing advertisements in a media guidanceapplication is discussed in greater detail in, for example, Knudson etal., U.S. patent application Ser. No. 10/347,673, filed Jan. 17, 2003,Ward, III et al. U.S. Pat. No. 6,756,997, issued Jun. 29, 2004, andSchein et al. U.S. Pat. No. 6,388,714, issued May 14, 2002, which arehereby incorporated by reference herein in their entireties. It will beappreciated that advertisements may be included in other media guidanceapplication display screens of the present invention.

Options region 126 may allow the user to access different types of mediacontent, media guidance application displays, and/or media guidanceapplication features. Options region 126 may be part of display 100 (andother display screens of the present invention), or may be invoked by auser by selecting an on-screen option or pressing a dedicated orassignable button on a user input device. The selectable options withinoptions region 126 may concern features related to program listings ingrid 102 or may include options available from a main menu display.Features related to program listings may include searching for other airtimes or ways of receiving a program, recording a program, enablingseries recording of a program, setting program and/or channel as afavorite, purchasing a program, or other features. Options availablefrom a main menu display may include search options, VOD options,parental control options, access to various types of listing displays,subscribe to a premium service, edit a user's profile, access a browseoverlay, or other options.

The media guidance application may be personalized based on a user'spreferences. A personalized media guidance application allows a user tocustomize displays and features to create a personalized “experience”with the media guidance application. This personalized experience may becreated by allowing a user to input these customizations and/or by themedia guidance application monitoring user activity to determine varioususer preferences. Users may access their personalized guidanceapplication by logging in or otherwise identifying themselves to theguidance application. Customization of the media guidance applicationmay be made in accordance with a user profile. The customizations mayinclude varying presentation schemes (e.g., color scheme of displays,font size of text, etc.), aspects of media content listings displayed(e.g., only HDTV programming, user-specified broadcast channels based onfavorite channel selections, re-ordering the display of channels,recommended media content, etc.), desired recording features (e.g.,recording or series recordings for particular users, recording quality,etc.), parental control settings, and other desired customizations.

The media guidance application may allow a user to provide user profileinformation or may automatically compile user profile information. Themedia guidance application may, for example, monitor the media the useraccesses and/or other interactions the user may have with the guidanceapplication. Additionally, the media guidance application may obtain allor part of other user profiles that are related to a particular user(e.g., from other web sites on the Internet the user accesses, such aswww.tvguide.com, from other media guidance applications the useraccesses, from other interactive applications the user accesses, from ahandheld device of the user, etc.), and/or obtain information about theuser from other sources that the media guidance application may access.As a result, a user can be provided with a unified guidance applicationexperience across the user's different devices. This type of userexperience is described in greater detail below in connection with FIG.4. Additional personalized media guidance application features aredescribed in greater detail in Ellis et al., U.S. patent applicationSer. No. 11/179,410, filed Jul. 11, 2005, Boyer et al., U.S. patentapplication Ser. No. 09/437,304, filed Nov. 9, 1999, and Ellis et al.,U.S. patent application Ser. No. 10/105,128, filed Feb. 21, 2002, whichare hereby incorporated by reference herein in their entireties.

Another display arrangement for providing media guidance is shown inFIG. 2. Video mosaic display 200 includes selectable options 202 formedia content information organized based on media type, genre, and/orother organization criteria. In display 200, television listings option204 is selected, thus providing listings 206, 208, 210, and 212 asbroadcast program listings. Unlike the listings from FIG. 1, thelistings in display 200 are not limited to simple text (e.g., theprogram title) and icons to describe media. Rather, in display 200 thelistings may provide graphical images including cover art, still imagesfrom the media content, video clip previews, live video from the mediacontent, or other types of media that indicate to a user the mediacontent being described by the listing. Each of the graphical listingsmay also be accompanied by text to provide further information about themedia content associated with the listing. For example, listing 208 mayinclude more than one portion, including media portion 214 and textportion 216. Media portion 214 and/or text portion 216 may be selectableto view video in full-screen or to view program listings related to thevideo displayed in media portion 214 (e.g., to view listings for thechannel that the video is displayed on).

The listings in display 200 are of different sizes (i.e., listing 206 islarger than listings 208, 210, and 212), but if desired, all thelistings may be the same size. Listings may be of different sizes orgraphically accentuated to indicate degrees of interest to the user orto emphasize certain content, as desired by the media provider or basedon user preferences. Various systems and methods for graphicallyaccentuating media listings are discussed in, for example, Yates, U.S.patent application Ser. No. 11/324,202, filed Dec. 29, 2005, which ishereby incorporated by reference herein in its entirety.

Users may access media content and the media guidance application (andits display screens described above and below) from one or more of theiruser equipment devices. FIG. 3 shows a generalized embodiment ofillustrative user equipment device 300. More specific implementations ofuser equipment devices are discussed below in connection with FIG. 4.User equipment device 300 may receive media content and data viainput/output (hereinafter “I/O”) path 302. I/O path 302 may providemedia content (e.g., broadcast programming, on-demand programming,Internet content, and other video or audio) and data to controlcircuitry 304, which includes processing circuitry 306 and storage 308.Control circuitry 304 may be used to send and receive commands,requests, and other suitable data using I/O path 302. I/O path 302 mayconnect control circuitry 304 (and specifically processing circuitry306) to one or more communications paths (described below). I/Ofunctions may be provided by one or more of these communications paths,but are shown as a single path in FIG. 3 to avoid overcomplicating thedrawing.

Control circuitry 304 may be based on any suitable processing circuitry306 such as processing circuitry based on one or more microprocessors,microcontrollers, digital signal processors, programmable logic devices,etc. In some embodiments, control circuitry 304 executes instructionsfor a media guidance application stored in memory (i.e., storage 308).In client-server based embodiments, control circuitry 304 may includecommunications circuitry suitable for communicating with a guidanceapplication server or other networks or servers. Communicationscircuitry may include a cable modem, an integrated services digitalnetwork (ISDN) modem, a digital subscriber line (DSL) modem, a telephonemodem, or a wireless modem for communications with other equipment. Suchcommunications may involve the Internet or any other suitablecommunications networks or paths (which is described in more detail inconnection with FIG. 4). In addition, communications circuitry mayinclude circuitry that enables peer-to-peer communication of userequipment devices, or communication of user equipment devices inlocations remote from each other (described in more detail below).

Memory (e.g., random-access memory, read-only memory, or any othersuitable memory), hard drives, optical drives, or any other suitablefixed or removable storage devices (e.g., DVD recorder, CD recorder,video cassette recorder, or other suitable recording device) may beprovided as storage 308 that is part of control circuitry 304. Storage308 may include one or more of the above types of storage devices. Forexample, user equipment device 300 may include a hard drive for a DVR(sometimes called a personal video recorder, or PVR) and a DVD recorderas a secondary storage device. Storage 308 may be used to store varioustypes of media described herein and guidance application data, includingprogram information, guidance application settings, user preferences orprofile information, or other data used in operating the guidanceapplication. Nonvolatile memory may also be used (e.g., to launch aboot-up routine and other instructions).

Control circuitry 304 may include video generating circuitry and tuningcircuitry, such as one or more analog tuners, one or more MPEG-2decoders or other digital decoding circuitry, high-definition tuners, orany other suitable tuning or video circuits or combinations of suchcircuits. Encoding circuitry (e.g., for converting over-the-air, analog,or digital signals to MPEG signals for storage) may also be provided.Control circuitry 304 may also include scaler circuitry for upconvertingand downconverting media into the preferred output format of the userequipment 300. Circuitry 304 may also include digital-to-analogconverter circuitry and analog-to-digital converter circuitry forconverting between digital and analog signals. The tuning and encodingcircuitry may be used by the user equipment to receive and to display,to play, or to record media content. The tuning and encoding circuitrymay also be used to receive guidance data. The circuitry describedherein, including for example, the tuning, video generating, encoding,decoding, scaler, and analog/digital circuitry, may be implemented usingsoftware running on one or more general purpose or specializedprocessors. Multiple tuners may be provided to handle simultaneoustuning functions (e.g., watch and record functions, picture-in-picture(PIP) functions, multiple-tuner recording, etc.). If storage 308 isprovided as a separate device from user equipment 300, the tuning andencoding circuitry (including multiple tuners) may be associated withstorage 308.

A user may control the control circuitry 304 using user input interface310. User input interface 310 may be any suitable user interface, suchas a remote control, mouse, trackball, keypad, keyboard, touch screen,touch pad, stylus input, joystick, voice recognition interface, or otheruser input interfaces. Display 312 may be provided as a stand-alonedevice or integrated with other elements of user equipment device 300.Display 312 may be one or more of a monitor, a television, a liquidcrystal display (LCD) for a mobile device, or any other suitableequipment for displaying visual images. In some embodiments, display 312may be HDTV-capable. Speakers 314 may be provided as integrated withother elements of user equipment device 300 or may be stand-alone units.The audio component of videos and other media content displayed ondisplay 312 may be played through speakers 314. In some embodiments, theaudio may be distributed to a receiver (not shown), which processes andoutputs the audio via speakers 314.

The guidance application may be implemented using any suitablearchitecture. For example, it may be a stand-alone application whollyimplemented on user equipment device 300. In such an approach,instructions of the application are stored locally, and data for use bythe application is downloaded on a periodic basis (e.g., from the VBI ofa television channel, from an out-of-band feed, or using anothersuitable approach). In another embodiment, the media guidanceapplication is a client-server based application. Data for use by athick or thin client implemented on user equipment device 300 isretrieved on-demand by issuing requests to a server remote to the userequipment device 300. In one example of a client-server based guidanceapplication, control circuitry 304 runs a web browser that interpretsweb pages provided by a remote server.

In yet other embodiments, the media guidance application is downloadedand interpreted or otherwise run by an interpreter or virtual machine(run by control circuitry 304). In some embodiments, the guidanceapplication may be encoded in the ETV Binary Interchange Format (EBIF),received by control circuitry 304 as part of a suitable feed, andinterpreted by a user agent running on control circuitry 304. Forexample, the guidance application may be a EBIF widget. In otherembodiments, the guidance application may be defined by a series ofJAVA-based files that are received and run by a local virtual machine orother suitable middleware executed by control circuitry 304. In some ofsuch embodiments (e.g., those employing MPEG-2 or other digital mediaencoding schemes), the guidance application may be, for example, encodedand transmitted in an MPEG-2 object carousel with the MPEG audio andvideo packets of a program.

User equipment device 300 of FIG. 3 can be implemented in system 400 ofFIG. 4 as user television equipment 402, user computer equipment 404,wireless user communications device 406, or any other type of userequipment suitable for accessing media, such as a non-portable gamingmachine. For simplicity, these devices may be referred to hereincollectively as user equipment or user equipment devices. User equipmentdevices, on which a media guidance application is implemented, mayfunction as a standalone device or may be part of a network of devices.Various network configurations of devices may be implemented and arediscussed in more detail below.

User television equipment 402 may include a set-top box, an integratedreceiver decoder (IRD) for handling satellite television, a televisionset, a digital storage device, a DVD recorder, a video-cassette recorder(VCR), a local media server, or other user television equipment. One ormore of these devices may be integrated to be a single device, ifdesired. User computer equipment 404 may include a PC, a laptop, atablet, a WebTV box, a personal computer television (PC/TV), a PC mediaserver, a PC media center, or other user computer equipment. WEBTV is atrademark owned by Microsoft Corp. Wireless user communications device406 may include PDAs, a mobile telephone, a portable video player, aportable music player, a portable gaming machine, or other wirelessdevices.

It should be noted that with the advent of television tuner cards forPCs, WebTV, and the integration of video into other user equipmentdevices, the lines have become blurred when trying to classify a deviceas one of the above devices. In fact, each of user television equipment402, user computer equipment 404, and wireless user communicationsdevice 406 may utilize at least some of the system features describedabove in connection with FIG. 3 and, as a result, include flexibilitywith respect to the type of media content available on the device. Forexample, user television equipment 402 may be Internet-enabled allowingfor access to Internet content, while user computer equipment 404 mayinclude a tuner allowing for access to television programming. The mediaguidance application may also have the same layout on the variousdifferent types of user equipment or may be tailored to the displaycapabilities of the user equipment. For example, on user computerequipment, the guidance application may be provided as a web siteaccessed by a web browser. In another example, the guidance applicationmay be scaled down for wireless user communications devices.

In system 400, there is typically more than one of each type of userequipment device but only one of each is shown in FIG. 4 to avoidovercomplicating the drawing. In addition, each user may utilize morethan one type of user equipment device (e.g., a user may have atelevision set and a computer) and also more than one of each type ofuser equipment device (e.g., a user may have a PDA and a mobiletelephone and/or multiple television sets).

The user may also set various settings to maintain consistent mediaguidance application settings across in-home devices and remote devices.Settings include those described herein, as well as channel and programfavorites, programming preferences that the guidance applicationutilizes to make programming recommendations, display preferences, andother desirable guidance settings. For example, if a user sets a channelas a favorite on, for example, the web site www.tvguide.com on theirpersonal computer at their office, the same channel would appear as afavorite on the user's in-home devices (e.g., user television equipmentand user computer equipment) as well as the user's mobile devices, ifdesired. Therefore, changes made on one user equipment device can changethe guidance experience on another user equipment device, regardless ofwhether they are the same or a different type of user equipment device.In addition, the changes made may be based on settings input by a user,as well as user activity monitored by the guidance application.

The user equipment devices may be coupled to communications network 414.Namely, user television equipment 402, user computer equipment 404, andwireless user communications device 406 are coupled to communicationsnetwork 414 via communications paths 408, 410, and 412, respectively.Communications network 414 may be one or more networks including theInternet, a mobile phone network, mobile device (e.g., Blackberry)network, cable network, public switched telephone network, or othertypes of communications network or combinations of communicationsnetworks. BLACKBERRY is a service mark owned by Research In MotionLimited Corp. Paths 408, 410, and 412 may separately or together includeone or more communications paths, such as, a satellite path, afiber-optic path, a cable path, a path that supports Internetcommunications (e.g., IPTV), free-space connections (e.g., for broadcastor other wireless signals), or any other suitable wired or wirelesscommunications path or combination of such paths. Path 412 is drawn withdotted lines to indicate that in the exemplary embodiment shown in FIG.4 it is a wireless path and paths 408 and 410 are drawn as solid linesto indicate they are wired paths (although these paths may be wirelesspaths, if desired). Communications with the user equipment devices maybe provided by one or more of these communications paths, but are shownas a single path in FIG. 4 to avoid overcomplicating the drawing.

Although communications paths are not drawn between user equipmentdevices, these devices may communicate directly with each other viacommunication paths, such as those described above in connection withpaths 408, 410, and 412, as well other short-range point-to-pointcommunication paths, such as USB cables, IEEE 1394 cables, wirelesspaths (e.g., Bluetooth, infrared, IEEE 802-11x, etc.), or othershort-range communication via wired or wireless paths. BLUETOOTH is acertification mark owned by Bluetooth SIG, INC. The user equipmentdevices may also communicate with each other directly through anindirect path via communications network 414.

System 400 includes media content source 416 and media guidance datasource 418 coupled to communications network 414 via communication paths420 and 422, respectively. Paths 420 and 422 may include any of thecommunication paths described above in connection with paths 408, 410,and 412. Communications with the media content source 416 and mediaguidance data source 418 may be exchanged over one or morecommunications paths, but are shown as a single path in FIG. 4 to avoidovercomplicating the drawing. In addition, there may be more than one ofeach of media content source 416 and media guidance data source 418, butonly one of each is shown in FIG. 4 to avoid overcomplicating thedrawing. (The different types of each of these sources are discussedbelow.) If desired, media content source 416 and media guidance datasource 418 may be integrated as one source device. Althoughcommunications between sources 416 and 418 with user equipment devices402, 404, and 406 are shown as through communications network 414, insome embodiments, sources 416 and 418 may communicate directly with userequipment devices 402, 404, and 406 via communication paths (not shown)such as those described above in connection with paths 408, 410, and412.

Media content source 416 may include one or more types of mediadistribution equipment including a television distribution facility,cable system headend, satellite distribution facility, programmingsources (e.g., television broadcasters, such as NBC, ABC, HBO, etc.),intermediate distribution facilities and/or servers, Internet providers,on-demand media servers, and other media content providers. NBC is atrademark owned by the National Broadcasting Company, Inc., ABC is atrademark owned by the ABC, INC., and HBO is a trademark owned by theHome Box Office, Inc. Media content source 416 may be the originator ofmedia content (e.g., a television broadcaster, a Webcast provider, etc.)or may not be the originator of media content (e.g., an on-demand mediacontent provider, an Internet provider of video content of broadcastprograms for downloading, etc.). Media content source 416 may includecable sources, satellite providers, on-demand providers, Internetproviders, or other providers of media content. Media content source 416may also include a remote media server used to store different types ofmedia content (including video content selected by a user), in alocation remote from any of the user equipment devices. Systems andmethods for remote storage of media content, and providing remotelystored media content to user equipment are discussed in greater detailin connection with Ellis et al., U.S. patent application Ser. No.09/332,244, filed Jun. 11, 1999, which is hereby incorporated byreference herein in its entirety.

Media guidance data source 418 may provide media guidance data, such asmedia listings, media-related information (e.g., broadcast times,broadcast channels, media titles, media descriptions, ratingsinformation (e.g., parental control ratings, critic's ratings, etc.),genre or category information, actor information, logo data forbroadcasters' or providers' logos, etc.), media format (e.g., standarddefinition, high definition, etc.), advertisement information (e.g.,text, images, media clips, etc.), on-demand information, and any othertype of guidance data that is helpful for a user to navigate among andlocate desired media selections.

Media guidance application data may be provided to the user equipmentdevices using any suitable approach. In some embodiments, the guidanceapplication may be a stand-alone interactive television program guidethat receives program guide data via a data feed (e.g., a continuousfeed, trickle feed, or data in the vertical blanking interval of achannel). Program schedule data and other guidance data may be providedto the user equipment on a television channel sideband, in the verticalblanking interval of a television channel, using an in-band digitalsignal, using an out-of-band digital signal, or by any other suitabledata transmission technique. Program schedule data and other guidancedata may be provided to user equipment on multiple analog or digitaltelevision channels. Program schedule data and other guidance data maybe provided to the user equipment with any suitable frequency (e.g.,continuously, daily, a user-specified period of time, a system-specifiedperiod of time, in response to a request from user equipment, etc.). Insome approaches, guidance data from media guidance data source 418 maybe provided to users' equipment using a client-server approach. Forexample, a guidance application client residing on the user's equipmentmay initiate sessions with source 418 to obtain guidance data whenneeded. Media guidance data source 418 may provide user equipmentdevices 402, 404, and 406 the media guidance application itself orsoftware updates for the media guidance application.

Media guidance applications may be, for example, stand-aloneapplications implemented on user equipment devices. In otherembodiments, media guidance applications may be client-serverapplications where only the client resides on the user equipment device.For example, media guidance applications may be implemented partially asa client application on control circuitry 304 of user equipment device300 and partially on a remote server as a server application (e.g.,media guidance data source 418). The guidance application displays maybe generated by the media guidance data source 418 and transmitted tothe user equipment devices. The media guidance data source 418 may alsotransmit data for storage on the user equipment, which then generatesthe guidance application displays based on instructions processed bycontrol circuitry.

Media guidance system 400 is intended to illustrate a number ofapproaches, or network configurations, by which user equipment devicesand sources of media content and guidance data may communicate with eachother for the purpose of accessing media and providing media guidance.The present invention may be applied in any one or a subset of theseapproaches, or in a system employing other approaches for deliveringmedia and providing media guidance. The following three approachesprovide specific illustrations of the generalized example of FIG. 4.

In one approach, user equipment devices may communicate with each otherwithin a home network. User equipment devices can communicate with eachother directly via short-range point-to-point communication schemesdescribe above, via indirect paths through a hub or other similar deviceprovided on a home network, or via communications network 414. Each ofthe multiple individuals in a single home may operate different userequipment devices on the home network. As a result, it may be desirablefor various media guidance information or settings to be communicatedbetween the different user equipment devices. For example, it may bedesirable for users to maintain consistent media guidance applicationsettings on different user equipment devices within a home network, asdescribed in greater detail in Ellis et al., U.S. patent applicationSer. No. 11/179,410, filed Jul. 11, 2005. Different types of userequipment devices in a home network may also communicate with each otherto transmit media content. For example, a user may transmit mediacontent from user computer equipment to a portable video player orportable music player.

In a second approach, users may have multiple types of user equipment bywhich they access media content and obtain media guidance. For example,some users may have home networks that are accessed by in-home andmobile devices. Users may control in-home devices via a media guidanceapplication implemented on a remote device. For example, users mayaccess an online media guidance application on a website via a personalcomputer at their office, or a mobile device such as a PDA orweb-enabled mobile telephone. The user may set various settings (e.g.,recordings, reminders, or other settings) on the online guidanceapplication to control the user's in-home equipment. The online guidemay control the user's equipment directly, or by communicating with amedia guidance application on the user's in-home equipment. Varioussystems and methods for user equipment devices communicating, where theuser equipment devices are in locations remote from each other, isdiscussed in, for example, Ellis et al., U.S. patent application Ser.No. 10/927,814, filed Aug. 26, 2004, which is hereby incorporated byreference herein in its entirety.

In a third approach, users of user equipment devices inside and outsidea home can use their media guidance application to communicate directlywith media content source 416 to access media content. Specifically,within a home, users of user television equipment 404 and user computerequipment 406 may access the media guidance application to navigateamong and locate desirable media content. Users may also access themedia guidance application outside of the home using wireless usercommunications devices 406 to navigate among and locate desirable mediacontent.

In some embodiments, user television equipment 402 includes aclickstream application or function 424. User computer equipment 404and/or wireless user communications device 406 may also include aclickstream application 426 and/or 428, respectively. Clickstreamapplication 426 and/or 428 may be part of clickstream application 424,or may be standalone clickstream applications. The following will referto clickstream application 424. However, it will be understood that thesame discussion may apply to clickstream applications 426 and/or 428. Insome embodiments, clickstream application 424 may include softwareand/or hardware components, partially or entirely implemented on usertelevision equipment 402, user computer equipment 404, wireless usercommunications device 406, media guidance data source 418, and/or anysuitable server(s). Clickstream application may monitor user actionswith the user television equipment 402, user computer equipment 404, orwireless user communications device 406. In addition, clickstreamapplication 424 may include analysis functions to analyze collected useractions.

It will be appreciated that while the discussion of media content ormedia items has focused on video content, the systems and methodsdescribed herein can be applied to other types of media content, such asmusic, images, podcasts, and the like. Other examples of media items mayinclude media programs, such as movie, television program, video, song,audio program, game, broadcast program, and multimedia program.

As described above, users may interact with the system via user inputinterface 310 in various ways, such as making a selection, or invoking acommand, etc. A clickstream application may capture any user interactionwith a media device (e.g., user equipment device 300, user televisionequipment 402, user computer equipment 404, and wireless usercommunications device 406) as clickstream data. Clickstream data mayinclude time-stamped indicators, where the time-stamped indicators maybe associated with at least a user identifier and a type of user actionwith a media device.

The following flow diagrams of FIGS. 5-7 illustrate various exemplaryprocesses involved in some aspects of the present disclosure. Whereappropriate, these processes may, for example, be implemented completelyor partially in the processing circuitry of a user equipment device(e.g., processing circuitry 306, user television equipment 402, usercomputer equipment 404 or wireless user communications device 406), orin a processing server located remotely from the user equipment device(e.g., media guidance data source 418). Illustrative examples of how theprocesses or variations of the processes may be practiced are discussedin further detail in relation to FIGS. 8-12.

Depending on how a user is using an input device, while consuming amedia item or interacting with a media application, information aboutthe user or groups of users (or the media item, if appropriate) may bededuced from the input device behavior as recorded in the clickstream.For example, user behavior with an input device may be indicative of auser's personality. In another example, the user behavior with an inputdevice during the consumption of a media item may be indicative of thedegree or level of user interest in a portion of a media item.

FIG. 5 shows a flow diagram of an exemplary method for deducing userinformation from user behavior. In some aspects, clickstream data may beused to gauge a user's degree or level of interest in a portion of amedia item. For instance, the speed of a user's presses on the volumebutton may be indicative of the degree or level of interest in theportion of a media item being consumed at the time (i.e., the faster thespeed, the higher the interest). In one illustrative example, process500 may be used to deduce information about a user's interest in aparticular portion of a media item.

First, clickstream application 424 receives a plurality of time-stampedindicators based on user actions over a period of time with a mediadevice while interacting with a media item (e.g., user equipment device300, user television equipment 402, user computer equipment 404, and/orwireless user communications device 406) (step 502). For example, usercomputer equipment 404, e.g. a personal computer, may includeclickstream application 424 that gathers and/or analyzes clickstreamdata captured from a user's interaction with user computer equipment 404and/or the like. In some instances, each of the time-stamped indicatorsare associated with a type of user action with the media device. Forexample, a user may press the “Ch+” (channel up) button on a remotecontrol during the consumption of a video. A signal may be sent from theremote control to a desktop computer. The desktop computer may trackuser actions such as this “Ch+” button press by storing a log of useractivities and/or actions (e.g., the “Ch+” button press may be logged asa time-stamped indicator associated with “Channel Up”). Similarly, theset-top box may forward the button press action information to a remoteserver, such as media guidance data source 418, for further processingand storage. While examples discussed herein are generally related toremote control button presses, one of ordinary skill in the art willunderstand that other kinds of user actions with other media and inputdevices (such as user input interface 310) may be received and/or storedas time-stamped indicators.

User actions recorded as time-stamped indicators may include at leastone of a keyboard stroke, mouse click, joystick command, keypaddepression, voice command, touchscreen touch, haptic interaction,gesture input, remote control key initiation, fast-forward command,rewind command, pause, stop, play, volume up, volume down, and set-topbox control command, and the like.

Depending on the system configuration and design, clickstreamapplication 424 may receive time-stamped indicators in real-time,periodically, continuously or at other time intervals. In some aspects,clickstream application 424 may receive time-stamped indicators atpre-scheduled time periods. For instance, time-stamped indicators may bereceived only during time periods of high activity (e.g., prime-timeperiod for television broadcasting). In another instance, clickstreamapplication 424 may receive time-stamped indicators from users/devicesthat have been opted in to having user actions tracked and monitored bythe system. Any suitable configuration for receiving or storingtime-stamped indicators may be used to optimize for system objectivessuch as protecting user privacy, improving system load and efficiency,and/or lowering errors in subsequent analyses. Analyses may includereports relating to, for example, viewer behavior, advertisingimpressions, audience measurements, feature usage and popularity,effectiveness of display structures, and other reports.

Log entries of user actions may include a list or array of time-stampedindicators, where each of the time-stamped indicators may include atleast one of: a time-stamp, a user identifier, a user action identifier,or any other suitable data fields useful for describing and analyzingtime-stamped user actions. Systems and methods for clickstream captureand analysis are described in detail in Milazzo, U.S. patent applicationSer. No. 12/570,778, filed Sep. 30, 2009, which is incorporated byreference herein in its entirety. Time-stamped indicators may be storedin a relational database, or in any suitable data structure thatprovides easy access to clickstream data, either locally at a userdevice or remotely at a remote server. Examples of time-stampedindicators are discussed in further detail with respect to FIGS. 9 and10 later herein.

In certain configurations, clickstream data and/or time-stampedindicators may originate from a plurality of media devices. Collectingclickstream behavior from multiple devices may create a more unifiedpicture of a user's behavior and actions. For example, a user mayinteract similarly with a cell phone as with a keyboard. Clickstreamapplication 424 may analyze clickstream behavior from both devices or aplurality of devices together for purposes of deducing information fromthe user behavior with a higher confidence.

The clickstream application 424 may monitor and analyze time-stampedindicators over a period of time as a user interacts with a media item,and/or monitor time-stamped indicators as a user interacts with a mediaapplication on a media device. For instance, the application 424 maygather time-stamped indicators based on a user's interactions with amobile phone, e.g., dialing, because these interactions may be relevantto determining the user's behavior while interacting with a media item.

A media item may include a media program. A media program may include atleast one of a movie, television program, video, song, audio program,game, broadcast program, and multimedia program. The media item may alsoinclude advertisements that promote products, services, person, company,or any other entity. Examples of advertisements include an imagepromotion, an interactive banner display on a webpage, an audioannouncement, a video clip, product placement within a media program, anicon displayed within a media program, and the like.

Once time-stamped indicators have been received, clickstream application424 can process the time-stamped indicators (i.e., raw clickstream data)for further analysis. The application 424 may derive data representativeof the raw clickstream data. In one configuration, clickstreamapplication 424 analyzes the clickstream data and examines the timebetween time-stamped indicators. In certain aspects, clickstreamapplication 424 examines the number of time-stamped indicators thatoccur within a period of time.

Clickstream application 424 may measure one or more time periods betweenthe time-stamped indicators and the number of time-stamped indicatorswithin a period of time (step 504). Clickstream application 424 maycalculate time period measurements by subtraction (or any suitablemethod) using processing circuitry, such as processing circuitry 306 orprocessing circuitry in a remote server or other media devices.Calculated measurements may be stored in a database or any suitablestorage.

The application 424 may include a predictive function that implements apredictive model to identify events, characteristics, and/or features ofa media item based on user behavior and/or clickstream data. Theapplication 424 may use a predictive model to identify characteristicsof a user or group of users based on clickstream data associated withtheir interactions with one or more media items. The predictive modeland/or algorithm may utilize statistics (e.g., a statistical model)and/or heuristics. In one configuration, certain patterns oftime-stamped indicators can be correlated with certain types of users orgroups of users. Patterns and their respective correlations with certainuser types may be defined manually, statistically, or based on anysuitable statistical, predictive, and/or artificial intelligence method.Based on the one or more of these methods, user information may bededuced from detected time-stamped indicators. Example methods includeBayesian inference, logical inference, Markov chain models, and thelike.

Various hypotheses and/or predictions may be made about user behavior bythe application 424. For instance, users with different personalitytypes may exhibit different clickstream patterns. Besides a user'sdegree of interest in a particular media item, other characteristics(e.g., a user's personality, language preferences, and physicalattributes) may be deduced from the observed time-stamped indicators orclickstream behaviors. For example, inferences may be made aboutphysical attributes, such as determining whether a user may be hearingimpaired, visually impaired, vocally impaired, mobility impaired,movement impaired, and/or cognitively impaired. In one configuration,long time periods between time-stamped indicators may indicate that auser is mobility impaired due to a lower than average clicking speed.

Such activity may also or alternatively indicate a casual oruninterested demeanor. In another embodiment, excessive use of the“volume up” control or high volume settings may indicate that a user ishearing impaired. Based on how closely an observed pattern matches anexpected or known pattern, a physical characteristic of the user may bededuced from the clickstream. For instance, a long time period onaverage between time-stamped indicators may indicate that a user isolder with a slower than average reaction time. In certainimplementations, an exact match to a defined pattern is not required.Rather, the application 424 uses a substantially close match to adefined and/or expected pattern to identify or infer user and/or mediaitem characteristics.

The system may identify user and/or media item characteristics based onobserved behavior from a plurality of users. Clustering techniques maybe used to group users into various groups and information may bededuced using such classification. For example, clustering techniquesmay create groups of users statistically based on a pre-defined distancefunction, where users who generally behave similarly may be groupedtogether into one group. Based on a set of time-stamped indicators andobserved behavior, the application 424 may classify a user into one ofthe groups. The application 424 may deduce information about a userbased at least in part on group membership.

Clickstream application 424 may determine a degree of user interest in aportion of the media item based on at least one time period between thetime-stamped indicators, the number of time-stamped indicators withinthe period of time, and/or at least one type of user action (step 506).As discussed above, in one illustrative configuration, a user whopressed the volume up button four times at a relatively fast pace whilewatching a news segment in a television program may indicate a strongerinterest in the news segment than a user who pressed the volume upbutton twice at a relatively slow pace (i.e., indicating a lower levelof interest). Any number of clickstream patterns may be used todetermine interest level or other kinds of user information.

In some aspects, clickstream application 424 may use other sources togather and/or deduce user information besides clickstream data andtime-stamped indicators. For example, user profile information may beleveraged to make a better inference and/or prediction on userinformation. User profile information may include user preferences,favorites, dislikes, likes, customizations, geographical location,biographical information, or other deduced information about the user.User profile information may or may not include information deduced fromclickstream data. In some configurations, the application 424 usestime-stamped indicators to predict user and/or media itemcharacteristics. For instance, a user who subscribes to the HomeShopping Network and frequently changes volume settings duringcommercials and/or advertisements may be identified as a prospectivecustomer and/or target for advertising promotions online and/or via amedia provider.

User profile information may be stored in a database locally on a user'smedia device or on a processing server (e.g., media guidance data source418) remote from the user. User profile information may be retrievedfrom user profiles via sources such as social media websites, onlinegaming profiles, third-party marketing databases, user-provided profileson media applications, and/or any other sources that store and gatheruser information.

In some configurations, information deduced about a user may be used todetermine the type of content associated with a portion of a media item.The application 424 may make the determination based on detected useractions over the period of time. For instance, if clickstream datacomprises time-stamped indicators of user disinterest (e.g., rapidtelevision channel surfing, quick successions of Fast Forward buttonpresses), the application 424 may determine that the content of theportion of the media item includes an infomercial, especially if theuser's profile indicates a general dislike of infomercials. Other typesof content may be associated with certain patterns of user actions, andappropriate information may be inferred therefrom based in part on theobserved clickstream.

In some configurations, information about or characteristics of aportion of a media item may be stored as metadata. The information maybe deduced from user behavior via monitoring of a clickstream associatedwith a user. The metadata, such as a description of the content, may bemanaged in a database such as a registry or repository. Metadata mayinclude an assortment of metatags, such as an identifier of the mediaitem, a description of the media content, tags/properties, a timingidentifier, or any suitable data fields proper for storing structureddata about a media item or portions of a media item. In certain aspects,the metadata is embedded in a media item. The metadata may be receivedfrom a media source, such as media content source 416 or media guidancedata source 418. The metadata may be embedded by the application 424 orby a media provider based on information from the application 424.

In some cases, metadata may be associated with a broadcast schedule. Ifthe metadata describes media content in relation to a broadcast schedule(such as television schedule program guide data, where the content isassumed to be consumed at the broadcast time), time-stamped indicatorsreceived during the consumption of a time-shifted media item (e.g.,stored prior to interacting with the media item over a period of time,or downloaded/buffered on-demand) may be processed to account for thetime-shift such that the time-stamped indicators may be correlated withthe metadata properly. The time-shifted media item may be recorded,buffered, delayed and/or stored, and the like.

In some configurations, the metadata may be independent from the time ofconsumption of the media item (e.g., metadata may be embedded in a videoprogram). The time-stamped indicators received during the consumption oftime-shifted media items may be correlated with identified portions ofthe media item (e.g., portions with associated time-independentmetadata) without accounting for the time-shift. Systems and methods fordeducing information about users during the consumption of time-shifteddata described above applies to process 600 and 700 of FIGS. 6 and 7respectively.

The application 424 may use metadata as part of its prediction processfor making an inference. In one configuration, the application 424 usesmetadata to change certain probabilities in a statistical predictionprocess. For instance, if a television program is tagged with genreinformation, the application 424 adjusts certain probabilities and/orreferences certain expected clickstream patterns based on the genremetadata where users are expected to, for example, behave differentlyduring a children's show versus a reality television program.

Besides deducing information about a user, the application 424 may useclickstream data for identifying extraordinary or salient moments andevents in a media item.

FIG. 6 shows an exemplary flow diagram of a method for identifying aportion of a media item. First, clickstream application 424 may receivea plurality of time-stamped indicators based on user actions over aperiod of time with a media device while the user interacts with a mediaitem (step 602). Details of step 602 are described with respect to step502 of FIG. 5. Second, any suitable method may be used to define and/orstore a plurality of behavior patterns based on time-stamped indicators(step 604). For example, media guidance data source 418 or any suitableserver accessible by clickstream application 424 may store definedbehavior patterns. Each behavior pattern may be associated with a set ofuser actions. In some configurations, the defined behavior patterns aredefined in terms of data representative of time-stamped indicators.

Behavior patterns may be defined based on empirical data. For example, astatistician and/or clickstream analyzer may analyze time-stampedindicators from a plurality of users for trends and patterns of userbehavior. Behavior patterns may be defined based on prior knowledgeabout users (e.g., based on prior social studies about human behavior,psychology research, and the like). Behavior patterns may beautomatically determined, in real time or off-line, based on predictiveand/or statistical algorithms that automatically associate certainclickstream data with known user characteristics and/or characteristicsof media items.

User behavior patterns may be defined based on certain patterns oftime-stamped indicators. For example, a sequence of button presses maybe used as a behavior pattern. Pattern definitions may include thenumber count of button presses, the speed of the button presses, thebutton or action performed, the application feature or media item beingselected, user characteristics associated with the pattern.

Clickstream application 424 may identify an event associated with amedia item by detecting a change in behavior pattern as a user interactswith the media item (step 606). For instance, it may be assumed thatusers may be in different activity and/or interest states depending onthe observed behavior. A user exhibiting slow or little activity may bein a disinterested state where the user may not be very excited aboutthe media item. At a certain point during consumption of a media item, auser's activity may change (e.g., a spike in the received time-stampedindicators), indicating that the user is excited about the content. Theuser has apparently transitioned from a disinterested state to anexcited/attentive state. The change in user behavior may indicate thelocation of an interesting event in a media item, such as, for example,a gossip news segment in an entertainment program. An event associatedwith the media item may be identified by detecting a type of behaviorpattern as the user interacts with the media item.

An event may be a commercial, an advertisement, a scene, a song, anoccurrence, an important message, an uninteresting portion, an offensiveportion, an extraordinary portion, a beginning of a media program, anend of a media program, a special appearance, a distorted portion, adisturbing portion, and the like. Typically, data about events in amedia item or media program is provided by third-party sources at acost. Using process 600, clickstream application 424 can identify eventswithout the need to pay an editor to manually label events in a mediaitem. Events may be derived automatically using a processor byunderstanding and monitoring user behavior and associated clickstreampatterns. In some instances, the application 424 using process 600 isadvantageously more effective at identifying interesting portions of amedia item (e.g., a show) because process 600 incorporates the realinterests/reactions of the population rather than relying on thepotentially subjective judgment of a human editor. The results ofprocess 600 may be more representative of the user population, andlikely of more value to interested parties such as content providers andadvertisers. In one configuration, rather than running a study group togauge the interest of individuals to a pilot television program, theapplication 424 monitors user behavior to identify the level or interestand extraordinary moments in the pilot television program based onprocess 600.

Metadata may be added that identifies the location of an event in amedia item. Metadata may include information about the event. Forexample, the metadata may label the event as a climax of a movie.Further details regarding metadata and its usage are described herein inrelation to FIG. 5.

The detection of changes in user behavior or detection of behaviorpatterns may be implemented using statistical models. A model of theuser may be used to describe user behavior in terms of time-stampedindicators, patterns and their associated probability distributions.According to one aspect, individuals are observed over time (e.g., byreceiving time-stamped indicators over a period of time), and alongitudinal model may be used, such as a Markov chain. According toanother aspect, users are observed in the aggregate, and a statisticalmodel of the population may be used, e.g., a Bayesian model.

In certain aspects, the identification of an event associated with themedia item comprises detecting an aggregate change in behavior patternrelated to a plurality of users as they interact with a media item. Athreshold may be set based on the number of users where the observedchange in behavior pattern occurs for the identification of an event.For example, an event may be identified if a total of 50 peopleexhibited similar changes in behavior patterns during the consumption ofthe same media item (e.g., an audio podcast). Users may be aggregatedand/or grouped based on age, sex, geography, or any suitable factors.

Besides identifying events in a media item, the application 424 mayderive other information about a media item using collaborative and/orsimilar filtering or predictive methods. For instance, if a plurality ofpeople have interacted with a portion of a media item in a similarfashion, the application 424 may deduce information about the portion ofthe media item based on characteristics of the plurality of users. Morespecifically, if a plurality of users exhibited behavior indicating ahigh level of interest in a portion of a video clip, information (e.g.,genre or type of targeted audience) about the video clip may be deduced.For example, if the users are fans of Britney Spears, the application424 may infer that the video clip was associated with Britney Spears. Amore detailed example is discussed later in relation to FIG. 12.

FIG. 7 shows a flow diagram of a method for generating information abouta media item. Clickstream application 424 may receive time-stampedindicators during the consumption of a media item from a plurality ofusers (step 702). Systems and methods for receiving time-stampedindicators from users are described in detail in relation to step 502 ofFIG. 5.

Clickstream application 424 may classify users into a plurality ofgroups based at least in part on the received time-stamped indicatorsfrom each user media device (step 704). Classification methods such assupervised learning and statistical classification may be used to modela population of users. Classification may include the process offiltering for information or patterns using techniques involvingtime-stamped indicators received from a plurality of users. Theapplication 424 may use clustering and/or classification techniques todeduce information about users in the aggregate, and/or to deduceinformation about media items being consumed.

Classification may include determining classes of users among apopulation. If desired, unsupervised learning may be used to determinehow the time-stamped indicators are organized without a set ofpre-defined classifications. To determine the classes among apopulation, the application 424 may use a clustering method to assign aset of observations (e.g., user behaviors, user profiles, time-stampedindicators) into subsets or clusters. The clustering method may includethe definition of a distance function that describes how similar atleast two users are based at least in part on their user profile ortime-stamped indicators. Based on the subsets/clusters, differentclasses and/or groups of users may be defined. Unsupervised learning maybe performed offline or online. Any suitable or similar learningmethods, such as neural networks, may be used for determining theclasses and/or groups of users among a population.

In some configurations, the application 424 uses supervised learningand/or reinforcement learning. A known set of classes, definitions,and/or associated data (e.g., pairs of time-stamped indicator withdesired classification) may be provided as training data to theclassification system. As such, the time-stamped indicators of a userwho has not been classified may be provided as inputs to theclassification system. The application 424, using the system, maypredict/learn the classification of a user. Any other suitablesupervised learning methods may be used to classify users among apopulation, such as, without limitation, support vector machines.

Results from the classification may be useful for deducing informationabout the media item being consumed. Users who behave a certain wayduring the consumption of a media item may provide insight about themedia item itself. According to one aspect, shared interests of userswho behaved similarly during the consumption of a media item may provideinformation about the genre of the media item. For example, users thatare classified as having an interest in antiques may show excitementduring a segment of the Today Show in the form of certain detectedclickstream behavior. This may indicate that the segment/portion of theToday Show is related to antiques.

The clickstream application 424 may derive information about a mediaitem based in part on a first characteristic shared among at least twoof the users belonging to a first group of users (step 706). Forexample, some or all users within a class/group may share an interest ingolf. This shared interest, along with the detection of similar behavioramong the group of users, may be indicative of the content in theportion of the media item (e.g., the content was about a golftournament, or golf players like this content). In another example, someor all of the users within the class/group may share a disinterest incertain content, e.g., dogs. The application 424 may deduce, based ondetected disinterested behavior among a group, that a portion of a mediaitem was associated with dogs. The application 424 may store thededuced/derived information about the media item as metadata.

Metadata associated with a media item may affect the classificationprocess. For instance, the type of media item (which may be indicated ina metatag) may affect the learning methods in the classificationprocess. In one configuration, the application 424 uses differentclassification methods depending on the type of media item. For example,the application 424 may use a supervised learning method for audio mediaitems while using an unsupervised learning method for video media items.In another configuration, the application 424 makes predictions and/orinferences based on the characteristics of the media item. Acharacteristic of a media item may be described by embedded and/orassociated metadata. For example, users with a shared interest in fishmay be interpreted differently for a show on the Food Network versus ashow on the Outdoor Channel. For the show on the Food Network,observations of a group of users with a shared interest in fish mayindicate that the media item is related to cooking fish. As for the showon the Outdoor Channel, the observations may indicate that the mediaitem is related to the sport of fishing instead.

The application 424 may associate shared characteristics with userprofiles of users in a group (e.g., stored as user metadata).Characteristics associated with user profiles may include preferences,customizations, favorites, properties derived from time-stampedindicators, and the like. For example, lists of favorite musicians in auser profile may be used as characteristics of a user. Users in a firstgroup may share the same liking in a particular musical artist.

Information about a media item may be derived from more than one of thegroups and/or classes identified by the classifying process. Theapplication 424 may derive information about a media item based in parton a second characteristic shared among users belonging to a secondgroup of users. For example, users of the second group may share aninterest in fly fishing which can be a shared characteristic. Thus, theapplication 424 may detect behavior of the first group to determine thata media item is related to the sport of fishing, while the detectedbehavior of the second group enables the application 424 to determinethat the media item is related to fly fishing.

Information about users and a media item derived from the time-stampedindicators may be advantageous in a recommendation system. Theapplication 424 may make a recommendation about a media item to a userof the first group based at least in part on a first characteristic. Forexample, if a shared characteristic of the first group is golf, theapplication 424 may recommend other golf related media items to users inthe first group. Media item recommendations may also include productsand services recommendations in commerce, and/or products and servicesshown in a media item.

According to another aspect, the application 424 may make a media itemrecommendation to a user of the second group based at least in part onthe first characteristic. For example, a media item related to golf(i.e., the first characteristic) may be recommended to users of thesecond group (e.g., the group of users who like fly fishing). In somecases both groups of users may have shown interest in the same mediaitem, as indicated by their time-stamped indicators. This recommendationmay be based on a prediction that users in the first group may agree oncertain interests with users in the second group (e.g., people who likefishing may also like golf or vice versa, if they both liked aparticular television program).

Collaborative filtering may enable a recommendation engine of theapplication 424 to cross recommend media items based on user and/orgroup characteristics. The recommendation may be made based in part onthe shared characteristic. The recommendation may be made based in parton an individual's user profile and/or group affiliation. Theapplication 424 may make a media item to a first user of a first groupbased at least in part on a user profile of a second user in the firstgroup. For instance, if a first user within a first group likes and/oris associated with poker, a poker tournament gaming website may berecommended to a second user within the first group, who may not haveindicated an interest in poker. Likewise, the metatag “poker” may beadded to the second user, based on a prediction that users who behavesimilarly may share common interests.

According to yet another aspect, if the metadata about a portion of amedia item (or just the media item) is known, that metadata may be addedto a profile associated with users of the first group. For example, ifthe portion of the media item has a metatag identifying the item asincluding content about cooking salmon, then a metatag “likes cookingsalmon” may be added to at least one profile of the users in the firstgroup.

The following figures and illustrations in FIGS. 8-12 are illustrativeexamples and applications of the processes 500, 600 and 700 as describedin relation to FIGS. 5-7.

Advertisers and marketing professionals are particularly interested indeducing user information from user behavior. FIG. 8 shows an exampleclickstream timeline associated with a user consuming media items. FIG.8 shows a timeline of a viewer's and/or user's activity between 7:00a.m. and 7:30 a.m. in the morning on channel 404 (section 802), digitalvideo recorder (section 804) and clickstream data (section 806). Theviewer turned on the television set 23 seconds after 7:01 (box 814) andturned up the volume less than a second after this moment just a tad(box 816). Then the viewer may have watched NBC's “Today Show” (box 808)quietly for about 5 minutes. Then suddenly, there is a burst of severalvolume-up requests.

Nielsen Ratings, the producers of the “Today Show,” and the show'sadvertisers may be interested in knowing what was happening in theprogram when the viewer pumped up the volume to hear it better. Theobservation by the application 424 that the viewer hit the volume-upbutton repeatedly is important, but how quickly the viewer hits the samebutton over-and-over-again may be important as well. In illustrativeFIG. 8, each volume-up button click (e.g., box 818) was around onesecond after the previous one. This cadence may indicate a substantial,but not unrestrained, excitement about the subject. For example, thesubject may be a news story about Tiger Woods. If the viewer had clickedthe volume-up button more frantically or more slowly, the viewer'sinterest level may be reassessed differently. Details about systems andmethods for determining a degree of user interest in a portion of amedia item are discussed in relation to FIG. 5.

According to one aspect, the application 424 can make additionalpredictions about the viewer's personality. Additional or otherpredictions may include: the viewer's identity and the viewer'sinterests based on the timings between the viewer's repeated buttonpushes. For instance, one click per second may suggest that the vieweris interested, but not extremely excited. The application 424 may alsobe able to determine and/or predict more fundamental aspects of a user'spersonality such as whether a user has a calm demeanor or is impulsive,along with other personality traits. Details about systems and methodsfor assessing information about personality or other usercharacteristics are discussed with respect to FIGS. 5, 9 and 10.

Other patterns of behavior may also provide insight about the viewerand/or the content being viewed. For example, the series of clicks inboxes 820 shows that the viewer has now tuned away from the Today Show.The application 424 may deduce that the user is no longer interested inthe Today Show, that the show has adjourned, or that the show has goneto commercial advertisements, or that something uninteresting hasstarted. Details about systems and methods for deducing informationabout the viewer and the media item are discussed further with respectto FIGS. 6, 7 and 9-12.

Another valuable burst of clicks in FIG. 8 may have occurred around7:21:25 a.m. (see boxes 822 and 824). At that time, the viewer hadswitched over to watching the movie “Psycho” (item 810) recorded in theearly hours of the same morning on the digital video recorder (DVR). Themovie gave way to a commercial break around 7:21:20 a.m. The viewer hitthe volume-down button two seconds later (see box 822). Then anotherburst of clicks happens (see box 824). The viewer hopped forward thirtyseconds three times through the commercials but then stopped. The viewerappears to have watched a commercial for about 15 seconds. Advertisement812 starting at around 2:31:36 a.m. during the recording day on themovie channel playing “Psycho” made an impression on the viewer almostfive hours later, a little before 7:22 a.m.

According to one configuration, a service provider may use suchinformation to charge advertisers for ads indicated to have been watchedin addition to or rather than charging advertisers for placingadvertisements in time slots regardless of whether anyone watches theadvertisements. By monitoring the clickstream data for changes inbehavior such as, for example, changes in certain click cadence, theapplication 424 can enable billing of advertisers based on verifiableviewer actions and/or interactions with media items.

In certain configurations, systems and methods for analyzing clickstreamdata are provided in which clickstream data is gathered by theapplication 424 that indicates user activity or inactivity on a mediadevice. Activity and/or inactivity may be indicative of whether a useractually consumed certain media items. The clickstream data may includea time-stamp of the user activity and/or the duration of the useractivity and/or inactivity. The clickstream application 424 may use avariable threshold associated with user inactivity (or activity) relatedto a media item. Media item characteristics may be used to determineand/or infer whether the media item (e.g., a television program, radioprogram, video clip, an advertisement, and the like) was actually viewedand/or consumed by a user. Such a threshold may differ from one mediaitem to another media item, if, for example, the media items havedifferent characteristics. The application 424 may use a variablethreshold associated with user inactivity during the time that a mediaitem is being broadcast to determine whether the media item was consumedby a user. For instance, the threshold for inactivity may be differentbetween the morning and the afternoon. User inactivity thresholds may bebased on, for example, a quantity and type of user interactivity,elapsed time between user interactions, elapsed time of inactivity, anda schedule associated with the media item. User activity and inactivitythresholds may be variable and based on third party data, group data,and/or user data.

Besides using time between inputs, other input patterns such as thenumber of interactions, the content selected or recorded, and/or theviewer's choices overall may be tracked. FIG. 8 shows tracking of useractivity over a period of half an hour. In some configurations, theapplication 424 tracks activity continuously and/or over any suitableperiods of time. The application 424 may allow users to opt in or optout of being tracked. Users may configure how and when they should betracked. For example, a user may choose to only be tracked duringcommercial breaks, and not while content is consumed. In anotherconfiguration, a user may choose to be tracked only on certain days ofthe week or time of the day. In yet another example, a user may chooseto be tracked only when the user is logged on to his/her personalprofile.

The application 424 may review various inputs such as clickstream dataas seen in FIG. 8, which may be compared and contrasted against anyother additional information about the user. The other additionalinformation may include other user profiles or activity data on otherplatforms. For example, other consumer electronic devices that a userinteracts with, such as music players, car electronics, and homeappliances, may be other sources of user data.

User behavior at the individual level may be gathered for multiple userssuch that aggregate/group behavior may be monitored. Systems and methodsmay be used to analyze how user groups react to certain advertisements,television shows, and/or media items. In another embodiment, inputbehavior across multiple individuals may be segmented by age, gender,temperament, attentiveness, or any other user characteristic. Moreinformation may be gathered about how many and what type of peoplerespond well or poorly to a given media item (e.g., television programs,video clips, advertisements). Feedback may be valuable to advertisersand media providers which sponsor and/or produce the media content.Details about systems and methods that monitor user behavior among aplurality of users are discussed with respect to FIGS. 6, 7, 11 and 12.

Depending on the analysis and inferences desired, the application 424may record user input device behavior and/or clickstream datadifferently. Input device behavior, such as clickstream data or logfiles of device activity, may include structured data sets havinginformation such as timestamps, and/or information about the nature ofthe input device behavior. Data fields may include a user identifier.The identifier may be used for purposes of tracking individual users.The application 424 may keep certain input device behavior anonymousand/or private so that user privacy may be protected depending on thedevice and/or user identifier.

Clickstream data may be supplied to a cable system headend, mediaserver, or other component that is configured to receive clickstreamdata from user equipment 300 or intermediate source for analysis.Clickstream data may be maintained and analyzed locally on userequipment 300. In either arrangement, clickstream data may be processedusing control circuitry 306 and stored in storage 308 on a permanent ortemporary basis. Clickstream data may include continuous collection ofany information relating to user activity. For example, any user inputinterface entries, such as remote-control key presses, channel changes,navigation and use of media guidance application features, recordinginformation, and/or other activity may be collected as clickstream data.In addition, clickstream data may also include information about a mediaguidance application and associated elements. Some examples of suchclickstream data may include a periodic health indication, availablefeatures, application configuration, element information, or otherinformation about the media guidance application.

The clickstream data may be analyzed as collected and sent to a remoteserver for analysis. In some embodiments, clickstream data may beprocessed, for example, using processing circuitry 306, to create datastructures or short sequences of data which may be referred to as logentries. The clickstream log entries may be stored in storage 308 orsent via a communications path to a remote storage device. Theclickstream log entries may also be sent to a headend or remote analysisfacility. When clickstream log entries are analyzed at a remote analysisfacility, clickstream log entries may be aggregated from multiple usersfor analysis. The analysis facility may store and process theclickstream logs and log entries and prepare various analytical reportsrelating to, for example, viewer behavior, advertising impressions,audience measurements, feature usage and popularity, effectiveness ofdisplay structures, and other reports.

FIG. 9 shows two example clickstream logs from two different users.Clickstream logs entries may include one or more devices and users. Inthis illustrative example, clickstream log 902 is associated with oneuser, and clickstream log 904 is associated with another user.Clickstream logs 902 and 904 (and clickstream logs 1002 and 1004 in FIG.10) may include data fields such as “User Id,” “Time-Stamp” and “Event.”“User Id” field allows the tracking of separate users, as long as the“User Id” is unique to each user (e.g., a randomly generated string, anIP address, or a MAC address). In some configurations, the “User Id” isassociated with a user profile, such as an on-line account associatedwith a web-site. In certain configurations, the data in the “User Id”field is associated with a device. In yet some other embodiments, thedata in the “User Id” field may change dynamically when the device orsystem detects that another user is using the device.

The “Time-Stamp” field may include any data suitable for tracking thetiming of each log entry in the clickstream. “Time-Stamp” field mayinclude a sequence of characters that denote the date and/or time atwhich an input device behavior event occurred, a counter, relative time,and the like. Data in “Time-Stamp” may be recorded by a computer and/orthe application 424 when the application 424 detects the occurrence ofinput device behavior and/or a user action. The timestamp may notcoincide exactly in time with the time in which the input devicebehavior, event, and/or activity occurs. Timing data may be logged in aconsistent format that enables efficient comparison of two differenttimestamp entries and tracking of progress over time. The format may bestandardized based on, without limitation, ISO 8601.

The “Event” field may include information about the nature of the inputdevice event. In this illustrative example, an identifier of the remotecontrol button press (e.g., “Fast Forward,” “Play,” “Volume Up,”“Rewind” and “Pause”) was recorded in the event field. Other inputdevice event information may include: type of device being used,keystroke inputs, features navigated, item selections, codes,abbreviations, or combinations thereof. Examples of such clickstreamdata may include a periodic health indication, application/device stateinformation, geography, origin/destination information, HTML requests,data transferred/received, data requested/submitted, error messages,available features, application configuration, element information,and/or other information about the media application and/or user device.

Each clickstream log entry or a group of clickstream log entries may bereferred to as time-stamped indicators that describes the timing andnature of an input device event. In some configurations, clickstreamsare stored as raw input device data. Clickstream data may be filtered,for example, by processing circuitry 406, such that certain lessimportant information or erroneous information may be removed. Filteringmay occur at processing circuitry 306, at a remote server, or a remotefacility using one or more filtering algorithms. Some examples ofclickstream information that may be filtered are periodic health statusindications that are normal, a routine pulse message, hardwareconfigurations, and/or other routine messages. Devices may be identifiedfor filtering based on random filter assignments, user relativeinteractivity, and/or information known about devices, such as location,type, user details, and other bases. Constant or consistent filteringalgorithms may be used in some embodiments. In other configurations,filtering algorithms may be provided that change, according to, forexample, user interactivity, hardware changes, media changes, and/orother basis.

As an example, clickstream logs 902 and 904 are compared to illustratehow user information may be deduced by the application 424. Sequence 906of clickstream log 902 shows a record of a user pressing on the “FastForward” button three times over a period of about one second, whilesequence 910 of clickstream log 904 shows a user pressing on the “FastForward” button twice with about 1.5 seconds in between. Sequence 906may be indicative of a user who wishes to fast forward past content thatthey dislike as quickly as possible, while sequence 910 may beindicative of a user who does not necessarily dislike the content andis, therefore, fast forwarding past the content at a slower pace.

Other deductions may be made, such as the age and/or age group of theuser. The application 424 may have a predictive rule that younger usersare quicker at using the fast forward feature because they have aquicker reaction time for knowing when to stop fast forwarding content(e.g., such as trying to skip advertisements during a commercial breakand stopping when a user senses that the television program is about toreturn from commercial break and pressing “Play”). If so, theapplication 424 may predict that a user exhibiting sequence 906 is morelikely to be a younger person than a user with sequence 910 (i.e.,someone who may be more savvy with digital video equipment). Thearrangement of sequence 910 being followed by sequence 912 (e.g., wherethe user pressed “Rewind” and “Play”) may indicate that the user'sreaction time was too slow and had missed the beginning of televisionprogram content after a commercial break. Systems and methods fordeducing user information are discussed in more detail with respect toFIG. 6.

According to another aspect, clickstream logs 902 and 904 may bevaluable for deducing information about the content of a media itembeing consumed by one or more users. For example, sequence 906 and/orsequence 910 may be indicative of an uninteresting portion of mediacontent (e.g., a commercial break) because the user did not seem to wantto consume the content. Instead, the user fast forwarded and skipped it.As such, uninteresting portions of media content and/or a media item maybe identified and located using clickstream data.

Similarly, the application 424 may identify and locatesalient/interesting portions in a media item. Sequence 908 may beindicative of an exciting moment in the media item being consumed,because a user has pressed “Volume Up” three times within a period ofabout a second. Behavior as such may indicate unrestrained interest inthe portion of the media item (e.g., FIG. 5). Sequence 908 may also beindicative of audio problems with the portion of the media content(e.g., the audio may be distorted, or recorded at a low volume).According to another aspect, sequence 908 may indicate that the user ishearing impaired. Sequence 912 may indicate that a user has found aninteresting portion of the media content because a user pressed rewindand paused the media item for a second review (e.g., a referee callduring a sports game). Related systems and methods for deducinginformation about the media content being consumed are discussed inrelation to FIGS. 6, 7, 10 and 11. Various information about how themedia content may be stored as metadata is discussed in relation to FIG.5.

Besides monitoring individual behavior, the application 424 mayaggregate and examine a plurality of user behaviors for the same mediaitem to analyze a media item being consumed. Aggregating clickstreamdata from multiple user devices may be valuable in identifying mediaconsumption trends and information about the content of a media item.Examples of valuable information deduced using aggregation ofclickstream data over many users may include identifying salientmoments/events in the media item, identifying different behavior of thepopulation based on time of day or genre of media being consumed,identifying popular portions of a media item, identifying unpopularportions of a media item, and so on. In some situations, aggregatebehavior may be more useful to media producers and advertisers when theymake a business decision to provide content to a large number of people(e.g., placing a superbowl advertisement or selecting a timeslot for atelevision broadcast program).

The application 424 may store aggregate behavior data and distributesuch data to producers and/or advertisers to enable them to tailor mediacontent based on user reaction and/or behavior. Aggregation ofclickstream data from multiple sources may also be used by theapplication 424 to better deduce information about the media item. Insome instances, clickstream data at the individual level may be noisy,intermittent, and/or inconsistent. The aggregation of clickstream dataamong multiple users can provide more definitive information about themedia item. Aggregation may be performed by the application 424 usingraw clickstream data or may occur with information derived from theclickstream data, or a combination of both.

FIG. 10 shows two additional example clickstream logs from two users.When examined together by the application 424, behavior patterns betweenthe user of clickstream log 1002 and the user of clickstream log 1004may indicate that there are certain extraordinary moments in the mediaitem being consumed. While the clickstreams 1002 and 1004 are from twousers consuming the same media item at the same time (e.g., a scheduledtelevision broadcast), similar methods may be applied to clickstreamdata for a media item that is consumed at different times by differentusers (e.g., on-demand or time-shifted media items). Systems and methodsfor relating clickstream data for users consuming the same media item atdifferent times are discussed further in relation to FIG. 5.

Sequences 1006 and 1010, although not exactly the same, may indicatethat a portion of the media content is uninteresting because both usersskipped the content at about the same relative time in the media itemplaying sequence. Sequences 1008 and 1014, although not in the exactsame fashion, show that the users both increased the volume (three timesfor clickstream log 1002 and twice for clickstream log 1014). Observinga plurality of users behaving in this manner may indicate that theportion of the media item is a salient event in the media item. At theindividual level, sequence 1012 of clickstream 1004 may indicate thatthe user was uninterested in the content consumed at 14:23. However,when information is aggregated over a plurality of users, theapplication 424 may determine that the media content is uninteresting tothe general population or uninteresting to a subset of the generalpopulation. In addition, information from a plurality of sources mayserve to validate clickstream information associated with individualusers.

FIG. 11 shows example clickstream logs from users in relation to a videoitem. The media item may include a plurality of different types ofcontent. In this illustration, FIG. 11 shows the clickstreams of peopleconsuming a show like “Saturday Night Live.” The show may includesegments such as “skit 1” 1101, ad 1102, “skit 2” 1003, band performance1104, “skit 3” 1105, ad 1106, “skit 4” 1107, band performance 1108, ad1109, “skit 5” 1110, and band performance 1111. The breakdown of theshow into segments may be stored as metadata. Various exemplary userbehavior and clickstreams received during the consumption of the mediaitem are shown in FIG. 11.

Based on aggregate user behavior, the application 424 may cluster and/orclassify users based on using methods or combinations of methodsdescribed in relation to FIGS. 5-7. Methods and systems described inFIGS. 5 and 6 may be used to detect user interest and certain behaviorpatterns based on individual behavior. Methods and systems described inFIG. 7 may be used to derive group behavior, behavior patterns, and/ormedia item information based on the group behavior. The clusteringprocess may include categorizing users with similar patterns of inputdevice behavior into separate groups. Clustering may be based on rawclickstream data (e.g., individual time-stamped indicators), or dataderived from the raw clickstream data (e.g., detected patterns from rawclickstream data).

According to one aspect, the application 424 detects a similaritybetween actions and/or users by at least defining distance functions. Adistance function may be defined in a way to determine how similar ordissimilar two clickstreams may be. In some aspects, the application 424compares a clickstream received with a defined and/or known clickstreampattern. In practice, two clickstreams may not be exactly the same.Therefore, the application 424 may use a distance function to findsimilarity between two slightly different clickstreams. A suitabledistance function may be defined in terms of one or more time-stampedindicators, respective components of one or more indicator (e.g.,timestamp, event, etc.), and/or data representative of the time-stampedindicators (e.g., time between clicks, number of clicks, etc.). Suitabledistance functions configured to compare two clickstreams mayincorporate Euclidean distance, Manhattan distance, Mahalanobisdistance, angle between two vectors, Hamming distance, and the like.

The application 424 may use a distance function in any suitableclustering algorithm, such as, without limitation, hierarchical,partitional, density-based, or subspace clustering methods. Anycombination of suitable methods may be used as well. Clusteringalgorithms may be useful because they can be configured to identifypatterns across a population of users. For instance, groups of users mayhelp identify what type of users are generally interested in theparticular portion of the media item. Clustering may also be helpful toidentify what type of content is in the portion of the media item beingconsumed.

In FIG. 11, during the beginning portion of ad 1102, the application 424records input device behavior from at least five different users. Threeof the five users (denoted in group 1114) may be grouped together basedon their similar behavior because, for example, they all exhibitedbehavior patterns associated with excitement. As discussed previously inrelation to the clickstream behavior of FIG. 8, multiple volume buttonpresses may indicate that someone has noticed something interesting andwanted to turn up the volume to hear it. A user request to record usingthe record button may also indicate that someone is so interested theywould like to record and rewatch it later. At the individual level, theapplication 424 may detect behavior patterns using methods and systemsdescribed in relation to FIGS. 5 and 6. In one configuration, theapplication 424 may create metadata based on detected collectivebehavior for that portion of the media item. For example, theapplication 424 may record the existence of an exciting moment in amedia item (e.g., a funny commercial with talking kittens) based on thedetected behavior of a plurality of users.

The other two of the five users (denoted in group 1115) may be groupedtogether because they both exhibited behavior patterns associated withdisinterest in the media item. Changing channels (i.e., pressing ChannelUp twice, or navigating to the guide to select other channels) mayindicate that the user is no longer interested in the content.Individually, or in aggregate, the input device behavior in group 1115may indicate that some type of user is not interested in the content atthat portion of the media item (e.g., determining that the two users areboth sports fanatics and both dislike that portion of the media item).In certain aspects, the group behavior in group 1115 may indicate a highprobability that undesirable content (e.g., advertisement, uninterestingcontent, etc.) is playing at the time. Similar information may also bederived or inferred by the application 424 from group behavior during ad1109 in group 1132.

During band performance 1104, at least six clickstreams have beenrecorded. Group 1116 may indicate user behavior exhibiting interest inthe content. If metadata about that portion of the media item isavailable, such as an identifier that U2 is the band performing duringthat portion of the media item, then the application 424 may addmetadata to the profiles of the users of group 1116 to indicate thatthose users have an interest in music or the band U2. During bandperformance 1111, group 1124 may indicate user behavior exhibitinginterest or excitement in the band playing during that segment.Clickstream behavior of users in group 1128, however, may indicatebehavior patterns exhibiting disinterest in that segment of the mediacontent. In a similar fashion, the application 424 may add metadata tothose user profiles indicating a disinterest in the band playing duringband performance 1111.

During a portion of “skit 4” 1107, at least six clickstreams have beenrecorded. Group 1122 may indicate that a group of users have found amoment or an event of the media item to be important. One user pausedand played the content. Another user pressed record. Yet another userpressed rewind and play. Using methods and systems described in FIG. 6,the application 424 may infer user behavior indicating a salient momentor event in a media item (e.g., a very funny moment during “skit 4”, ora very embarrassing moment, or a special appearance by a celebrity, andthe like). Furthermore, the application 424 may add derived informationabout the media content, such as the location of the salient event inthe media content or what kind of users enjoy the content, as metadata.On the other hand, users in group 1124 may indicate that some users arenot interested in that portion of the media content, perhaps finding itoffensive. The application 424 may record this negative reaction and addthis as metadata to the media item. If details about the salient eventare known, metadata may be added to the users exhibiting a reaction suchthat a disinterest in that particular event is associated with the usersof group 1124.

FIG. 12 shows example clickstream logs from users in relation to a mediaitem. Using methods and systems described in FIG. 5, the application 424may group users based on clickstream behavior detected during theconsumption of media item 1202. Using methods and systems described inFIG. 7, users may be grouped using any suitable clustering method.

In one configuration, user input device behavior collected from users1204, 1206 and 1208 by the application 424 may indicate a collectiveinterest in a portion of a media item. Using detection methods describedin FIG. 6, a level of interest may be derived by the application 424based on the clickstream behavior. If available, information about theportion of interest in media item 1202 may be derived collaboratively bythe application 424 based on the user profile information for users1204, 1206 and 1208. For example, users 1204 and 1206 may be sportsfans. The shared characteristic of the two users may indicate thatsports fans in general are users who may be interested in that portionof media item 1202. This indication may be added as metadata tag 1214.In certain configurations, other content known to be related to sportsmay be recommended to users 1204, 1206 and 1208. Due to the sharedinterest of users 1204, 1206 and 1208 in that portion of the mediacontent, the application 424 may infer that users 1204, 1206 and 1208share similar interests as well. Personal favorites of users 1204, 1206and 1208 may be recommended to each other. For instance, the application424 may recommend a portion of the favorites of user 1208 to user 1204.

Similarly, during a different portion of media item 1224, theclickstream behaviors of users 1218, 1220 and 1222 may indicate that agroup of users are interested in that portion of media item 1202. Theshared interest of users 1218, 1220 and 1222 in the celebrity Bono mayindicate that users who like Bono may generally find that portion ofmedia item 1202 interesting. The application 424 may add this indicationto media item 1202 as metadata 1224. In certain embodiments, becauseusers 1204, 1206, 1208, 1218, 1220, and 1222 all share interest in mediaitem 1202, the application 424 may infer that each user is likely toappreciate the favorite content that each of the other users enjoys. Forinstance, the favorite content of user 1206 may be recommended to user1218, 1220, and/or 1222. In another instance, the application 424 mayrecommend general content related to sports to user 1218, 1220, and/or1222. Because users 1218, 1220 and 1222 share the same interest incomedy, other content related to comedy may be recommended by theapplication 424 to user 1204, 1206 and/or 1208.

Besides deriving information about positive reactions to media item1202, negative reactions may also be useful in deriving informationabout media item 1202. During the same portion of media item 1202, theapplication 424 may use the clickstream behavior of users 1210 and 1212to identify a disinterest in that portion of media item 1202.Collaboratively, the shared interest in Sci-Fi of users 1210 and 1212may indicate that Sci-Fi lovers in general do not like that portion ofmedia content 1202. The negative information about the group of users1210 and 1212 may be added as metadata using tag 1216 to indicate thatthe portion of the content should not be targeted to Sci-Fi lovers.

The above described embodiments of the present invention are presentedfor purposes of illustration and not of limitation, and the presentinvention is limited only by the claims which follow.

What is claimed is:
 1. A method for providing media assetrecommendations, the method comprising: receiving data that identifies:(a) quantity of user interactions with a first media asset for a groupof users, and (b) elapsed time between the user interactions with thefirst media asset for the group of users; determining an inactivitythreshold for the first media asset based on a) the quantity of the userinteractions with the first media asset for the group of users, and (b)the elapsed time between the user interactions with the first mediaasset for the group of users; receiving a plurality of time-stampedindicators from a media device, wherein each of the plurality oftime-stamped indicators is associated with a corresponding actionperformed by a user with the media device with respect to the firstmedia asset that is being generated for display; identifying, based onthe plurality of time-stamped indicators and the corresponding useraction, periods of user activity and inactivity with respect to thefirst media asset by comparing frequency of plurality of time-stampedindicators to the inactivity threshold; determining based on theidentified periods of user activity and inactivity whether the userprefers the first media asset, wherein determining whether the userprefers the first media asset further comprises: determining a behaviorpattern of the user based on the plurality of time-stamped indicatorsreceived from the media device, wherein determining the behavior patternof the user further comprises: calculating a number of time-stampedindicators received over a period of time; calculating a speed of theuser action corresponding to each of the time-stamped indicatorsreceived over the period of time; determining a type of the user actioncorresponding to each of the time-stamped indicators received over theperiod of time; and determining the behavior pattern based at least inpart on the number of time-stamped indicators, the speed of the useraction corresponding to each of the time-stamped indicators, and thetype of the user action corresponding to each of the time-stampedindicators; and determining that the behavior pattern indicates a userpreference for the characteristic of the first media asset; in responseto determining that the user prefers the first media asset: retrieving acharacteristic of the first media asset; determining that thecharacteristic of the first media asset matches a first characteristicassociated with a user group among the group of users that is associatedwith a plurality of predetermined media preferences; in response todetermining that the characteristic of the first media asset matches thefirst characteristic associated with the user group, adding the userinto the user group; retrieving a second characteristic associated withthe user group; retrieving a second media asset having the secondcharacteristic; and generating for display, to the user, a media assetrecommendation of the second media asset.
 2. The method according toclaim 1, wherein determining that the behavior pattern indicates a userpreference for the characteristic of the first media asset furthercomprises: determining, based on metadata associated with the firstmedia asset, a plurality of segments of the first media asset;determining, based on the time-stamped indicators, that the user actioncorresponds to a first segment of the plurality of segments; anddetermining, based on the behavior pattern, that the user has apreference for a characteristic of the first segment of the plurality ofsegments.
 3. The method according to claim 1, wherein the user actionperformed with the media device with respect to the first media assetcomprises a command to modify playback of the first media asset.
 4. Themethod according to claim 1, wherein determining that the characteristicof the first media asset matches the first characteristic associatedwith the user group further comprises: retrieving the firstcharacteristic associated with the user group; and comparing thecharacteristic of the first media asset with the first characteristicassociated with the user group.
 5. The method according to claim 4,wherein retrieving the first characteristic associated with the usergroup further comprises retrieving a characteristic shared by each of aplurality of users in the user group.
 6. The method according to claim1, further comprising: receiving a second plurality of time-stampedindicators from media devices associated with each of a plurality ofusers in the user group, wherein each of the second plurality oftime-stamped indicators is associated with a corresponding actionperformed by the user with the media devices with respect to a thirdmedia asset; determining, based on the user action, that the pluralityof users in the user group prefer the third media asset; and assigning acharacteristic associated with each of the plurality of users in theuser group to the third media asset.
 7. The method according to claim 6,wherein determining that the plurality of users in the user group preferthe third media asset further comprises: determining, based on metadataassociated with the third media asset, a plurality of segments of thethird media asset; determining, based on the plurality of time-stampedindicators, that the user action corresponds to a first segment of theplurality of segments of the third media asset; and determining that theuser action indicates that the plurality of users in the user groupprefer a characteristic of the first segment of the plurality ofsegments of the third media asset.
 8. The method according to claim 1further comprising adjusting the inactivity threshold for the firstmedia asset based on a time of day.
 9. A system for providing mediaasset recommendations, the system comprising: control circuitryconfigured to: receive data that identifies: (a) quantity of userinteractions with a first media asset for a group of users, and (b)elapsed time between the user interactions with the first media assetfor the group of users; determine an inactivity threshold for the firstmedia asset based on a) the quantity of the user interactions with thefirst media asset for the group of users, and (b) the elapsed timebetween the user interactions with the first media asset for the groupof users; receive a plurality of time-stamped indicators from a mediadevice, wherein each of the plurality of time-stamped indicators isassociated with a corresponding action performed by a user with themedia device with respect to the first media asset that is beinggenerated for display; identify, based on the plurality of time-stampedindicators and the corresponding user action, periods of user activityand inactivity with respect to the first media asset by comparingfrequency of plurality of time-stamped indicators to the inactivitythreshold; determine based on the identified periods of user activityand inactivity whether the user prefers the first media asset, wherein,when determining whether the user prefers the first media asset, thecontrol circuitry is further configured to: determine a behavior patternof the user based on the plurality of time-stamped indicators receivedfrom the media device, wherein, when determining the behavior pattern ofthe user, the control circuitry is further configured to: calculate anumber of time-stamped indicators received over a period of time;calculate a speed of the user action corresponding to each of thetime-stamped indicators received over the period of time; determine atype of the user action corresponding to each of the time-stampedindicators received over the period of time; and determine the behaviorpattern based at least in part on the number of time-stamped indicators,the speed of the user action corresponding to each of the time-stampedindicators, and the type of the user action corresponding to each of thetime-stamped indicators; and determine that the behavior patternindicates a user preference for the characteristic of the first mediaasset; in response to determining that the user prefers the first mediaasset: retrieve a characteristic of the first media asset; determinethat the characteristic of the first media asset matches a firstcharacteristic associated with a user group among the group of usersthat is associated with a plurality of predetermined media preferences;in response to determining that the characteristic of the first mediaasset matches the first characteristic associated with the user group,add the user into the user group; retrieve a second characteristicassociated with the user group; retrieve a second media asset having thesecond characteristic; and generate for display, to the user, a mediaasset recommendation of the second media asset.
 10. The system accordingto claim 9, wherein, when determining that the behavior patternindicates a user preference for the characteristic of the first mediaasset, the control circuitry is further configured to: determine, basedon metadata associated with the first media asset, a plurality ofsegments of the first media asset; determine, based on the time-stampedindicators, that the user action corresponds to a first segment of theplurality of segments; and determine, based on the behavior pattern,that the user has a preference for a characteristic of the first segmentof the plurality of segments.
 11. The system according to claim 9,wherein the user action performed with the media device with respect tothe first media asset comprises a command to modify playback of thefirst media asset.
 12. The system according to claim 9, wherein, whendetermining that the characteristic of the first media asset matches thefirst characteristic associated with the user group, the controlcircuitry is further configured to: retrieve the first characteristicassociated with the user group; and compare the characteristic of thefirst media asset with the first characteristic associated with the usergroup.
 13. The system according to claim 12, wherein, when retrievingthe first characteristic associated with the user group, the controlcircuitry is further configured to retrieve a characteristic shared byeach of a plurality of users in the user group.
 14. The system accordingto claim 9, wherein the control circuitry is further configured to:receive a second plurality of time-stamped indicators from media devicesassociated with each of a plurality of users in the user group, whereineach of the second plurality of time-stamped indicators is associatedwith a corresponding action performed by the user with the media deviceswith respect to a third media asset; determine, based on the useraction, that the plurality of users in the user group prefer the thirdmedia asset; and assign a characteristic associated with each of theplurality of users in the user group to the third media asset.
 15. Thesystem according to claim 14, wherein, when determining that theplurality of users in the user group prefer the third media asset, thecontrol circuitry is further configured to: determine, based on metadataassociated with the third media asset, a plurality of segments of thethird media asset; determine, based on the plurality of time-stampedindicators, that the user action corresponds to a first segment of theplurality of segments of the third media asset; and determine that theuser action indicates that the plurality of users in the user groupprefer a characteristic of the first segment of the plurality ofsegments of the third media asset.
 16. The system according to claim 9wherein the control circuitry is further configured to adjust theinactivity threshold for the first media asset based on a time of day.